Anti-Bot Bypass

retry_on_block — Server-side retry

Auto-retry up to 3 times with a different IP and browser fingerprint on CAPTCHA or 403. Only charges credits on the successful attempt.

result = client.scrape(
    "https://protected-site.com",
    browser=True,
    retry_on_block=True,
)

Extended timeout

When retry_on_block=True, the SDK automatically extends the timeout to 300 seconds to accommodate server-side retries.

Resource blocking

Block unnecessary resources to speed up page load and reduce detection surface. Requires browser=True.

Block by resource type

result = client.scrape(
    "https://heavy-site.com",
    browser=True,
    block_resources=["image", "font", "media", "stylesheet"],
)

Valid types: image, stylesheet, font, media, script, document, xhr, fetch.

Block by URL pattern

Block requests whose URL contains any of the substrings (case-insensitive):

result = client.scrape(
    "https://example.com",
    browser=True,
    block_requests=["google-analytics", "doubleclick", "googletagmanager"],
)

Combined — optimal for hard sites

result = client.scrape(
    "https://www.google.com/shopping/...",
    browser=True,
    block_resources=["image", "font", "media"],
    block_requests=["google-analytics", "doubleclick", "googlesyndication"],
    retry_on_block=True,
)

The API picks the best engine for each target domain automatically. You don't need to specify which browser — just ask for browser=True and the server routes the request to the right engine.

Response guidance

Every response includes a guidance object that tells you what happened and what to try next:

result = client.scrape("https://hard-site.com")

if not result.guidance.success:
    print(result.guidance.error_type)         # "captcha", "timeout", "login_wall", etc.
    print(result.guidance.error_provider)      # "cloudflare", "datadome", etc.
    print(result.guidance.next_steps)          # ["Try with browser=true", ...]
    print(result.guidance.suggested_request)   # ready-to-use params dict

    if result.guidance.stop_reason:
        print("Don't retry:", result.guidance.stop_reason)
    elif result.guidance.suggested_request:
        # Retry with the suggested parameters
        result = client.scrape(**result.guidance.suggested_request)

See Error Handling for the full guidance reference.

Strategy guide

Site difficulty	Recommended approach
Easy (static HTML, no protection)	client.scrape(url)
Medium (JS rendering needed)	client.scrape(url, browser=True)
Medium (soft blocks, Cloudflare)	browser=True, retry_on_block=True
Hard (Google, Amazon)	browser=True, retry_on_block=True, block_resources=[...], block_requests=[...]
Login walls, geo-blocked	Check guidance.stop_reason — retrying won't help